Vehicle system and vehicle

ABSTRACT

A vehicle system includes a warning device and at least one electronic control unit. The at least one electronic control unit is configured to present a light stimulus to a driver, and determine whether the driver is facing forward, determine whether the driver holds a steering wheel, and cause the warning device to perform a first warning when it is determined that the driver is not facing forward while the light stimulus is being presented. The first warning is either the vibration or the sound. The at least one electronic control unit is configured to perform a second warning when it is determined that the driver does not hold the steering wheel before a predetermined time elapses after the first warning is started. The second warning includes the vibration and the sound.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to Japanese Patent Application No.2016-225084 filed on Nov. 18, 2016, which is incorporated herein byreference in its entirety.

BACKGROUND

1. Technical Field

The present disclosure relates to a vehicle system and a vehicle.

2. Description of Related Art

Japanese Patent Application Publication No. 2015-138383 (JP 2015-138383A) describes a device that includes a head-up display using thewindshield in front of a driver. This device blinks the two positionsthat are spaced equally to the left and to right of the position infront of the driver and are within the driver's peripheral visual range.In this way, the device keeps the driver awake.

SUMMARY

However, when the wakefulness level of a driver is reduced to someextent or when the driver relies on autonomous driving so much that thedriver's interest in driving is reduced, there is a possibility that thedriver cannot respond, or feels troublesome to react, to a lightstimulus. The device described in Japanese Unexamined Patent ApplicationPublication No. 2015-138383, if is used for such a driver, does notproduce a great effect, sometimes resulting in a reduced speed ofdriver's response to a change in the traffic environment.

The present disclosure provides a vehicle system and a vehicle that canencourage a driver to pay attention to driving according to the state ofthe driver.

A first aspect of the disclosure provides a vehicle system. The vehiclesystem according to the first aspect includes: at least one of avibration generation device configured to generate a vibration and asound output device configured to output a sound, and a displayprojection device configured to project a display on a display area of ahead-up display provided in a vehicle. The at least one electroniccontrol unit configured to present a light stimulus to a driver of thevehicle by using the head-up display when a change in a trafficenvironment is detected ahead of the vehicle, while the vehicle isperforming autonomous driving, determine whether the driver is lacingforward, determine whether the driver holds a steering wheel, and causethe warning device to perform a first warning, when it is determinedthat the driver is not facing forward while the light stimulus is beingpresented, and perform a second warning, when it is determined that thedriver does not hold the steering wheel before a first predeterminedtime elapses after the first warning is started. The first warning iseither the vibration generated by the vibration generation device or thesound output by the sound output device. The second warning includes thevibration generated by the vibration generation device and the soundoutput by the sound output device.

According to the above aspect, this vehicle system enables the driverwho is aware of driving to pay attention ahead. Moreover, the aboveaspect allows the vehicle system to give a stimulus also to a non-visualsense of the driver who is not facing forward while a light stimulus isbeing presented, thus encouraging the driver to face forward. Inaddition, this vehicle system can give the second warning, which isstronger than the first warning, to a driver whose wakefulness level isfurther reduced or to a driver who relies on autonomous driving so muchthat the interest in driving is further reduced. In this way thisvehicle system can encourage the driver to pay attention to drivingaccording to the state of the driver.

In the first aspect of the disclosure, the at least one electroniccontrol unit may be configured to increment the number of re-holdingtimes, when it is determined that the driver faces forward before asecond predetermined time elapses after the second warning is started orwhen it is determined that the driver holds the steering wheel beforethe second predetermined time elapses after the second warning isstarted, and output a signal for terminating the autonomous driving ofthe vehicle to an autonomous driving electronic control unit, when thenumber of re-holding times while the light stimulus is being presented,is equal to or greater than a threshold. The autonomous drivingelectronic control unit may be provided in the vehicle and may beconfigured to perform the autonomous driving of the vehicle.

There is a tendency that a driver is overconfident in autonomous drivingif the number of times the driver re-holds the steering wheel inresponse to the second warning is equal to or greater than thethreshold. Such a driver tends to engage in a task ether than drivingand have no intention to pay attention to driving. According to theabove aspect, if there is a tendency that the driver has no intention topay attention to driving, this vehicle system can terminate theautonomous driving of the vehicle to force the driver to drive thevehicle himself or herself.

In the first aspect of the disclosure, the at least one electroniccontrol unit may be configured to decelerate the vehicle or stop thevehicle when it is determined that the driver does not hold the steeringwheel before the second predetermined time elapses after the secondwarning is started.

If a driver does not re-hold the steering wheel in response to thesecond warning, there is a possibility that wakefulness level of thedriver is extremely low. According to the above aspect, this vehiclesystem can decelerate or stop the vehicle to ensure traveling safety.

In the first aspect of the disclosure, the at least one electroniccontrol unit may be configured to decelerate the vehicle or to stop thevehicle when it is determined that the driver does not hold the steeringwheel before a second predetermined time elapses after the secondwarning is started.

According to the above aspect, this vehicle system can decelerate orstop the vehicle to ensure traveling safety.

In the first aspect of the disclosure, the vehicle system may include adriver monitor camera. The first determination unit may be configured todetect whether the driver is facing forward by using the driver monitorcamera.

In the first aspect of the disclosure, the second warming may have astimulus intensity higher than the stimulus intensity of the firstwarning, or excite more senses of the driver than the first warning.

A second aspect of the disclosure provides A vehicle. The vehicleaccording to the second aspect includes: a sensor configured to detect achange in a traffic environment ahead of the vehicle; a head-up displayincluding & display projection device, the display projection devicebeing configured to present a light stimulus to a driver of the vehicleby projecting a display on a display area of the head-up display basedon a detection result of the sensor; a warning device configured tooutput at least one of a vibration and a sound; and at least oneelectronic control unit configured to cause the display projectiondevice to present the light stimulus, determine whether the driver isfacing forward, cause the warning device to perform a first warning whenit is determined that the driver is not facing forward while the lightstimulus is being presented, and determine whether the driver holds asteering wheel and, when it is determined that the driver does not holdthe steering wheel before a first predetermined time elapses after thefirst warning is started, cause the warning device to perform a secondwarning. The second warning has a stimulus intensity higher than thestimulus intensity of the first warning, or excites more senses of thedriver than the first warning.

In the second aspect of the disclosure, the vehicle system may includean autonomous, driving electronic control unit configured to performautonomous driving of the vehicle. The at least one electronic controlunit may be configured to cause the autonomous driving electroniccontrol unit to perform the autonomous driving of the vehicle, incrementthe number of re-holding times when it is determined that the driverfaces forward before a second predetermined time elapses after thesecond warning is started or when it is determined that the driver holdsthe steering wheel before the second predetermined time elapses afterthe second warning is started, and output a signal for terminating theautonomous driving of the vehicle to the autonomous driving electroniccontrol unit when the number of re-holding times is equal to or greaterthan a threshold.

In the second aspect of the disclosure, the at least one electroniccontrol unit may be configured to decelerate the vehicle or to stop thevehicle when it is determined that the driver does not hold the steeringwheel before a second predetermined time elapses after the secondwarning is started.

According to one aspect and the embodiments of the present disclosure,the vehicle system and a vehicle that can encourage a driver to payattention to driving according to the state of the driver is provided.

BRIEF DESCRIPTION OF THE DRAWINGS

Features, advantages, and technical and industrial significance ofexemplary embodiments of the disclosure will be described below withreference to the accompanying drawings, in which like numerals denotelike elements, and wherein:

FIG. 1 is a functional block diagram of a vehicle including a vehiclesystem according to a first embodiment;

FIG. 2A is a diagram showing the projection of blinking displays on awindshield;

FIG. 2B is a diagram showing a depression angle formed when a driverviews the lower end of a preceding vehicle from the driver's eye point;

FIG. 3 is a diagram showing blinking displays projected on thewindshield;

FIG. 4 is a plan view showing a squeezing candidate vehicle and apreceding vehicle;

FIG. 5 is a diagram showing the projection of a first blinking display;

FIG. 6 is a flowchart of light stimulus presentation start processing;

FIG. 7 is a flowchart of light stimulus presentation terminationprocessing;

FIG. 8 is a flowchart of first warning start processing;

FIG. 9 is a flowchart of first warning termination processing;

FIG. 10 is a functional block diagram of a vehicle including a vehiclesystem according to a second embodiment;

FIG. 11 is a functional block diagram of a vehicle including a vehiclesystem according to a third embodiment;

FIG. 12 is a flowchart of first warning termination processing andsecond warning start processing;

FIG. 13 is a flowchart of second warning termination processing;

FIG. 14 is a functional block diagram of a vehicle including a vehiclesystem according to a fourth embodiment;

FIG. 15 is a flowchart of light stimulus presentation terminationprocessing and count reset processing;

FIG. 16 is a flowchart of second warning termination processing,counting processing, and emergency processing; and

FIG. 17 is a flowchart of autonomous driving termination processing.

DETAILED DESCRIPTION OF EMBODIMENTS

Embodiments will be described in detail below with reference to thedrawings. In the description below, the same reference numeral is usedfor the same or equivalent component and the duplicated description isomitted.

First Embodiment

[Outline of Vehicle System 1]

FIG. 1 is a functional block diagram of a vehicle 2 including a vehiclesystem 1 according to a first embodiment. The vehicle system 1 ismounted on the vehicle 2 such as a passenger car. The vehicle 2 is avehicle capable of autonomous driving. Autonomous driving refers tovehicle control in which the vehicle 2 is autonomously driven along atarget route that is set in advance. During autonomous driving, thevehicle 2 travels autonomously with no need for the driver to performthe driving operation. The target route is a route on the map on whichthe vehicle 2 will travel in the autonomous driving mode.

The vehicle system 1 presents a light stimulus to the driver of thevehicle 2 using the head-up display of the vehicle 2 when a change inthe traffic environment is detected ahead of the vehicle 2 duringautonomous driving. The traffic environment includes the travelingsituation of the vehicles around the vehicle 2, the signal lightingsituation, the road condition, and so on. A change in the trafficenvironment means a change in the traffic environment from the trafficenvironment at a reference time. A change in the traffic environment isdetected, for example, when there is a change: to the extent that thedriver is required to pay attention. A specific example of a change inthe traffic environment is that the preceding vehicle has deceleratedits speed at a rate higher than the threshold speed or that anothervehicle has squeezed into the space ahead of the vehicle 2.

The driver's driving awareness (consciousness or interest concerning thedriving of the vehicle) may decrease during autonomous driving. Forexample, since the driver's driving operation is not required duringautonomous driving, the wakefulness level of the driver tends to reducemore easily during autonomous driving than during manual operation. Inaddition, during autonomous driving, the driver may make a call by amobile phone, watch the side scenery, play a game on a mobile terminal,or read a book. In other words, during autonomous driving, the drivermay be engaged in a task other than driving.

The vehicle system 1 projects a display on the windshield of the vehicle2 to give a fight stimulus to the driver for reducing a decline indriving awareness. A light stimulus is a visual stimulus, for example, ablinking display. A light stimulus also has au effect on the driver'speripheral visual field. For this reason, a light stimulus functionseffectively not only for a driver whose face is oriented to the frontdirection but whose wakefulness level is reducing but also for a driverwho is looking aside. A light stimulus, if presented to a driver who isaware of driving, allows the driver to improve the reduced wakefulnesslevel or to turn the face toward the front to try to recognize thetraffic environment. In addition, to prevent the driver from feelingannoying by excessive light stimulus presentation, the vehicle system 1terminates the light stimulus presentation when the driver faces thefront or when the driver turns the face toward the front in response tothe light stimulus. That is, when the driver's face is orientated to thefront while a light stimulus is being presented, the vehicle system 1terminates the light stimulus presentation assuming that a driver isable to respond appropriately.

On the other hand, when the driver does not turn the face toward thefront in response to a light stimulus, the vehicle system 1 determinesthat the driver's wakefulness level has reduced to such an extent thatthe driver cannot respond appropriately to an environmental change ordetermines that the driver relies on autonomous driving so much that heor she is engaged in another task. In this case, a light stimulus is notso effective and, therefore, the vehicle system 1 stimulates a senseother than a visual sense depending on the state of such a driver tohelp the driver to pay attention to driving. That is, the vehicle system1 gives an appropriate stimulus according to the state pf the driver toencourage the driver to pay attention to driving.

[Configuration of Vehicle System 1]

As shown in FIG. 1, the vehicle 2 includes an autonomous driving ECU 3(an example of an autonomous driving control unit), a stereo camera 4, aradar sensor 5, a driver monitor camera 6, a vehicle speed sensor 7, anElectronic Control Unit (ECU) 10, and a warning unit 20.

The autonomous driving ECU 3 is an electronic control unit having acomputing unit such as a Central Processing Unit (CPU), storage unitssuch as a Read Only Memory (ROM) and a Random Access Memory (RAM), and aController Area Network (CAN) communication circuit. The function of theautonomous driving ECU 3, which will be described later, is realizedwhen the CPU of the autonomous driving ECU 3 loads a program, stored inthe storage device, and executes the loaded program.

The autonomous driving ECU 3 performs autonomous driving. The autonomousdriving ECU 3 is connected to a map database (not shown) that stores mapinformation, a positioning unit (not shown) that determines the positionof the vehicle 2 on the map by the Global Positioning System (GPS),various sensors (not shown) that detect the traveling state of thevehicle 2, and various actuators (not shown) that cause the vehicle 2 totravel.

The map information stored in the map database includes information onthe position of a road (position information for each lane), informationon the shape of a road (for example, information on whether the road isa curved road or a straight road, the curvature of a curve, etc.),information on a road width (information on a lane width), informationon the gradient of a road, information on the cant angle of a road, andinformation on the limited vehicle speed on a road. The various sensorsinclude a yaw rate sensor that detects the yaw rate of the vehicle 2 andan acceleration sensor that detects the acceleration of the vehicle 2.The various actuators include a steering actuator that controls thesteering angle of the vehicle 2, a brake actuator that controls thebrake system of the vehicle 2, and an engine actuator that controls theengine of the vehicle 2 (or the motor of an electric vehicle).

The autonomous driving ECU 3 searches for a target route, from thecurrent position of the vehicle 2 to the destination, based on the mapinformation stored in the map database, the position information of thevehicle 2 on the map positioned by a positioning unit such as a sensor,and the destination that is set in advance. The autonomous driving ECU 3generates a travel plan for allowing the vehicle 2 to travel along thetarget route. The travel plan includes, for example, a steering targetvalue and a vehicle speed target value that are set for eachpredetermined distance on the target route. The autonomous driving ECU 3uses a known method to generate a travel plan. The autonomous drivingECU 3 autonomously drives the vehicle 2 according to the travel planbased on the position information on the vehicle 2 on the map positionedby the positioning unit. The autonomous driving ECU 3 sends controlsignals to various actuators to control the vehicle 2 for autonomouslydriving the vehicle.

The stereo camera 4 is a capturing device for capturing the image of thearea ahead of and behind the vehicle 2. The stereo camera 4 has twocapturing cameras arranged to reproduce the disparity between the righteye and the left eye. For example, the two capturing cameras areprovided on the vehicle 2, one on the interior side of the frontwindshield and the other on the interior side of the rear windshield.The stereo camera 4 sends the captured information on the area ahead ofand behind the vehicle 2 to the ECU 10. The captured informationgenerated by the stereo camera 4 also includes the information on thedepth direction.

The radar sensor 5, provided, for example, at the front end of thevehicle 2, uses radio waves (or light) to detect an obstacle ahead ofthe vehicle 2 (including an obstacle obliquely in front of the vehicle2). The radar sensor 5 detects an obstacle by sending radio waves in theforward direction of the vehicle 2 then receiving radio waves reflectedby the obstacle such as another vehicle. The radar sensor 5 sends theobstacle information on the detected obstacle to the ECU 10.

The driver monitor camera 6, provided, for example, on the cover of thesteering column of the vehicle 2 and in the position in front of thedriver, captures the image of the driver's face (see FIG. 2A). FIG. 2Ashows the capturing range Dp of the driver monitor camera 6. The drivermonitor camera 6 sends the captured information on the driver Dr to theECU 10.

The vehicle speed sensor 7 is a detector that detects the vehicle speedof the vehicle 2. An example of the vehicle speed sensor 7 is a wheelspeed sensor that is provided on the wheels of the vehicle 2, or on thedrive shaft that rotates in synchronization with the wheels, to detectthe rotation speed of the wheels. The vehicle speed sensor 7 sends thedetected vehicle speed information to the ECU 10.

The ECU 10 is an electronic control unit having a computing device suchas a CPU, storage devices such as a ROM and a RAM, a CAN communicationcircuit, and so on. The function of the ECU 10, which will be describedlater, is realized when the CPU of the ECU 10 loads a program, stored inthe storage device, and executes the loaded program. The ECU 10 isconnected to the autonomous driving ECU 3, stereo camera 4, radar sensor5, driver monitor camera 6, vehicle speed sensor 7, and warning unit 20.

The warning unit 20, mounted on the vehicle 2, sends the information tothe driver. The warning unit 20 includes a display projection unit 201and a sound output unit 202. The display projection unit 201 is a devicethat projects a display on the display area included in the peripheralvisual field of the driver. One example of the display projection unit261 is a part of the configuration of the head-up display that projectsa display of various information on the windshield. The displayprojection unit 201 projects a display of various information on thewindshield based on control signals received from the ECU 10.

FIG. 2A is a diagram showing the projection of blinking displays on thewindshield. FIG. 2A shows the driver monitor camera 6, the displayprojection unit 201, a driver Dr, a ground line G corresponding to theground, a height Eh of a driver's eye point Ep, a straight line Hpextending in the longitudinal direction of the vehicle 2 via thedriver's eye point Ep, a light stimulus P, a straight line Hu joiningthe driver's eye point Ep and the upper end of the light stimulus P, anangle θe formed by the straight line Hp and the straight line Hu, and adistance Lp from the driver's eye point Ep to lire front end of thevehicle 2. The driver's eye point Ep is, for example, a virtual point(one point) representing the position of the eyes of the driver Dr inthe normal driving state. The drivers eye point Ep is determined, forexample, to be a preset position in the cabin of the vehicle 2. Theposition of the driver's eye point Ep is determined, for example, whenthe vehicle 2 is designed or shipped. FIG. 2B will be described later.

FIG. 3 is a diagram showing blinking displays projected on thewindshield. FIG. 3 shows a windshield W of the vehicle 2, a travelinglane (host vehicle traveling lane) R1 in which the vehicle 2 travels, aleft-side white line L1 of the traveling lane R1, a right-side whiteline L2 of the traveling lane R1, an adjacent lane R2 adjacent to theleft side of the traveling lane R1, another vehicle (corresponding to asqueezing candidate vehicle that will be described later) Na travelingin the adjacent lane R2, a preceding vehicle Nb of the vehicle 2, anouter display frame C1 and an inner display frame C2, a referencehorizontal line D, a first blinking display Pa (light stimulus P) forcausing the driver to pay attention to the other vehicle Na, and asecond blinking display Pb (light stimulus P) for causing the driver topay attention to the preceding vehicle Nb. The first blinking display Paand the second blinking display Pb are virtual images projected on thewindshield W. FIG. 3 will be described later more in detail.

The sound output unit 202, for example, an in-vehicle speaker, is adevice that outputs sound.

Next, the functional configuration of the ECU 10 will be described. TheECU 10 includes an another-vehicle recognition unit 101, a white linerecognition unit 102, a vehicle speed determination unit 103, asqueezing candidate vehicle determination unit 104, a preceding vehicledetermination unit 105, a warning control unit 106, and a firstdetermination unit 107.

The another-vehicle recognition unit 101 recognizes another vehiclearound the vehicle 2 based on the information captured by the stereocamera 4 and/or the obstacle information sensed by the radar sensor 5.The another-vehicle recognition unit 101 recognizes the relativeposition of the other vehicle with respect to the vehicle 2 and themoving direction and moving speed of the other vehicle with respect tothe vehicle 2.

The white line recognition unit 102 recognizes the left and right whitelines L1 and L2, which form the traveling lane R1, based on theinformation captured by the stereo camera 4. The white line recognitionunit 102 uses a known image processing method to recognize the whitelines based on the image captured by the stereo camera 4. The white linerecognition unit 102 may recognize the white lines L1 and L2 using thereflection data on the light or radio waves, sensed by the radar sensor5, based on the obstacle information sensed by the radar sensor 5.

The vehicle speed determination unit 103 determines whether the vehiclespeed of the vehicle 2 is equal to or greater than the vehicle speedthreshold based on the vehicle speed information sensed by the vehiclespeed sensor 7. The vehicle speed threshold is a threshold that is setin advance (for example, 30 km/h). The vehicle speed determination unit103 determines the vehicle speed of the vehicle 2 because an excessivevisual stimulus, if given to the driver when the vehicle 2 is travelingslowly or is stationary, is sometimes annoying.

The squeezing candidate vehicle determination unit 104 determinesWhether there is a squeezing candidate vehicle that will squeeze infront of the vehicle 2 if it is determined by the vehicle speeddetermination unit 103 that the vehicle speed of the vehicle 2 is equalto or greater than the vehicle speed threshold. The squeezing candidatevehicle determination unit 104 determines whether there is a squeezingcandidate vehicle based on the recognition result of the another-vehiclerecognition unit 101. Squeezing means that another vehicle squeezesbetween the vehicle 2 and the preceding vehicle. Squeezing includes acase in which, though there is no preceding vehicle, another vehiclesqueezes in front of the vehicle 2 (for example, within 10 m from thevehicle 2).

FIG. 4 is a plan view showing a squeezing candidate vehicle and apreceding vehicle. FIG. 4 shows the vehicle 2, the left-side adjacentlane R2 and a right-side adjacent lane R3 both of which are adjacent tothe traveling lane R1, a squeezing candidate vehicle Na. and thepreceding vehicle Nb. As shown in FIG. 4, an example of the squeezingcandidate vehicle Na is another vehicle that travels in the adjacentlane R2 (or adjacent lane R3) adjacent to the traveling lane R1, inwhich the vehicle 2 travels, and that satisfies a determinationcondition that is set in advance. The determination condition may be setbased on the following factors: the lateral distance between the othervehicle and the traveling lane R1 of the vehicle 2 (the distance in thewidth direction of the traveling lane R1), the relative speed betweenthe other vehicle and the vehicle 2, whether or not the directionindicator of the other vehicle is turned on, and the inter-vehicledistance between the vehicle 2 and the preceding vehicle Nb. Theadjacent lane R2, though a merging lane that merges into the travelinglane R1 in FIG. 4, need not be a merging lane.

The squeezing candidate vehicle determination unit 104 determines thatthere is the squeezing candidate vehicle Na if the lateral distancebetween another vehicle and the traveling lane R1 of the vehicle 2 isequal to or smaller than the lateral distance threshold and if therelative speed between the other vehicle and the vehicle 2 is equal toor smaller than the relative speed threshold. The lateral distancebetween the other vehicle and the traveling lane R1 of the vehicle 2,which is the distance in the lane width direction of the traveling laneR1 in plan view, may be the distance between the end of the othervehicle on the traveling lane R1 side (the left or right end on thetraveling lane R1 side) and the white line L1 of the traveling lane R1(the white line on the other vehicle side). The relative speed betweenthe other vehicle and the vehicle 2 is the absolute value of therelative speed between the other vehicle and the vehicle 2 in thetraveling direction of the vehicle 2. The lateral distance threshold isa value (for example, 1 m) that is set in advance. The relative speedthreshold is also & value (for example, 3 km/h) that is set in advance.Usually, the squeezing candidate vehicle Na matches its speed to thespeed of the vehicle 2 (in this case, the relative speed becomes closeto zero) before actually squeezing in front of the vehicle 2 in order tocheck the state of the vehicle 2. Considering this behavior of thesqueezing candidate vehicle Na, the squeezing candidate vehicledetermination unit 104 determines that the relative speed between thesqueezing candidate vehicle Na and the vehicle 2 is equal to or smallerthan the relative speed threshold. It should be noted that the squeezingcandidate vehicle determination unit 104 may determine another vehiclein the adjacent lane traveling in parallel with, or in front of, thevehicle 2 as being the squeezing candidate vehicle Na. In addition, thesqueezing candidate vehicle determination unit 104 does not necessarilyneed to determine the relative speed between another vehicle and thevehicle 2; instead, the squeezing candidate vehicle determination unit104 may determine whether there is the squeezing candidate vehicle Nabased only on the determination result of the lateral distance betweenthe other vehicle and the traveling lane R1.

The squeezing candidate vehicle determination unit 104 determines thatthere is the squeezing candidate vehicle Na when the direction indicatoron the traveling lane R1 side of another vehicle is turned on. Thesqueezing candidate vehicle determination unit 104 uses a known imageprocessing method to determine whether the direction indicator on thetraveling lane R1 side of another vehicle is turned on based on theinformation captured by the stereo camera 4.

The squeezing candidate vehicle determination unit 104 determines thatthere is no squeezing candidate vehicle Na if the inter-vehicle distancebetween the vehicle 2 and the preceding, vehicle Nb is equal to orsmaller than the inter-vehicle distance threshold. The inter-vehicledistance threshold is a value (for example, 1 m) that is set in advance.If the inter-vehicle distance between the vehicle 2 and the precedingvehicle Nb is short, the squeezing candidate vehicle determination unit104 determines that there is no squeezing candidate vehicle Na becausethere is no room for another vehicle to squeeze in front of the vehicle2.

The preceding vehicle determination unit 105 determines whether, beforethe vehicle 2, there is the preceding vehicle Nb whose deceleration isequal to or greater than the deceleration threshold if it is determinedby the vehicle speed determination unit 103 that the vehicle speed ofthe vehicle 2 is equal to or greater than the vehicle speed threshold.The preceding vehicle Nb is a vehicle that is traveling, immediatelybefore the vehicle 2, in the same traveling lane as that of the vehicle2 (see FIG. 4). As shows in FIG. 4, based on the recognition result ofthe another-vehicle recognition unit 101 (for example, a change in thespeed of the preceding vehicle Nb detected by the radar sensor 5), thepreceding vehicle determination unit 105 determines whether there is thepreceding vehicle Nb whose deceleration is equal to or greater than thedeceleration threshold. The deceleration threshold is a value that isset in advance. The deceleration threshold may be a value that variesaccording to the inter-vehicle distance between the vehicle 2 and thepreceding vehicle Nb. The deceleration threshold may be a value that issmaller as the inter-vehicle distance between the vehicle 2 and thepreceding vehicle Nb (or the time to collision) is smaller.

The first determination unit 107 determines whether or not the driver isfacing forward. For example, the first determination unit 107 determineswhether or not the driver is facing forward based on the image capturedby the driver monitor camera 6. The first determination unit 107 uses aknown image processing method to make the determination described above.The first determination unit 107 outputs the determination result to thewarning control unit 106.

The warning control unit 106 causes the warning unit 20 to present alight stimulus. The warning control unit 106 outputs a signal to thewarning unit 20 to control the operation of the display projection unit201 of the warning unit 20, thereby causing the display projection unit201 to present a light stimulus to the driver. In this way, the warningcontrol unit 106 can stimulate the vision of the driver.

A specific example of how the warning control unit 106 controls thepresentation of a light stimulus will be described. If it is determinedby the squeezing candidate vehicle determination unit 104 that there isthe squeezing candidate vehicle Na, the warning control unit 106 causesthe display projection unit 201 to project the first blinking display Paon the windshield W (see FIG. 3). The first blinking display Pa is alight stimulus tor alerting the driver to the squeezing candidatevehicle Na.

If it is determined that there is the squeezing candidate vehicle Na,the warning control unit 106 recognizes the image of the squeezingcandidate vehicle Na on the windshield W, as viewed by the driver Drfrom the driver's eye point Ep, based on the recognition result of theanother-vehicle recognition unit 101. For example, based on the imagecaptured by the stereo camera 4, the warning control unit 106 uses aknown image processing method (viewpoint conversion processing or thelike) to recognize the image of the squeezing candidate vehicle Na.

The warning control unit 106 causes the display projection unit 201 toproject the first blinking display Pa within the outer display frame C1,which has a circular shape shown in FIG. 3, in such a manner that theline of sight of the driver Dr is guided toward the image of thesqueezing candidate vehicle Na. The outer display frame C1 is a frame ofa size large enough to include the effective visual field of the driverDr when viewing the area ahead of the vehicle 2 from the driver's eyepoint Ep. The effective visual field is a range that the driver Dr canvisually and instantly recognize a target, such as a preceding vehicle,only by eye movement with focus on the target. For example, theeffective visual field is a range of an upward angle of 8°, a downwardangle of 12°, and a right and left depression angle of 15°. The outerdisplay frame C1 may be an elliptical frame.

The reference horizontal line D shown in FIG. 3 is a virtual horizontalline extending right and left from the image of the preceding vehicleNb. The warning control unit 106 causes the display projection unit 201to display the first blinking display Pa in an area below the referencehorizontal line D in the outer display frame C1. The outer display frameC1 and the reference horizontal line D are not projected on thewindshield W. The outer display frame C1 and the reference horizontalline D need not necessarily be set.

The warning control unit 106 causes the display projection unit 201 toproject the first blinking display Pa at a position obliquely below theimage of the squeezing candidate vehicle Na (below the image and closeto the traveling lane R1) within the outer display frame C1 on thewindshield W. For example, the warning control unit 106 causes thedisplay projection unit 201 to display the first blinking display Pa ata position that is closer to the side of the adjacent lane R2 than thewhite line L1 on the side of the adjacent lane R2 in which the squeezingcandidate vehicle Na is traveling. For example, the warning control unit106 causes the display projection unit 201 to project the first blinkingdisplay Pa at a position on the windshield W with a downward depressionangle of 3° and a leftward or rightward depression angle of 4°. Thewarning control unit 106 causes the display projection unit 201 todisplay the first blinking display Pa in such a way that the firstblinking display Pa does not overlap with the image of the squeezingCandidate vehicle Na.

FIG. 5 is a diagram showing the projection of the first blinking displayPa. In FIG. 5, the squeezing candidate vehicle Na is traveling in theright-side adjacent lane R3 of the vehicle 2. In FIG. 5, the expectedfuture squeezing candidate vehicle Na is indicated by a broken line. Theattention guidance range Cpa is a range to which the attention (line ofsight) of the driver, who is looking ahead, is guided by the firstblinking display Pa. The attention guidance range Cpa may be a range ofa circle centered at the first blinking display Pa on the windshield Wwith a radius of a fixed distance.

As shown in FIG. 5, the warning control unit 106 may cause the displayprojection unit 201 to project the first blinking display Pa on themovement path of the image of the squeezing candidate vehicle Na on thewindshield W (the movement path as viewed by the driver from thedriver's eye point Ep). The warning control unit 106 causes the displayprojection Unit 201 to project the first blinking display Pa on themoving path of the squeezing candidate vehicle Na on the assumption thatthe image of the squeezing candidate vehicle Na on the windshield W willmove to the position close to the traveling lane R1 (the positionindicated by the broken line in FIG. 5). In this way, the warningcontrol unit 106 causes the display projection unit 201 to project thefirst blinking display Pa on the movement path of the image of thesqueezing candidate vehicle Na. This makes it easy to project the firstblinking display Pa so that the image of the squeezing candidate vehicleNa in the future is included in the attention guidance range Cpa, thusenhancing robustness (display stability). It should be noted that thewarning control unit 106 may cause the display projection unit 201 toproject the first blinking display Pa below the movement path.

Next, the projection of the second blinking display Pb by the warningcontrol unit 106 will be described. If it is determined by the precedingvehicle determination unit 105 that, before the vehicle 2, there is thepreceding vehicle Nb whose deceleration is equal to or greater than thedeceleration threshold, the warning control unit 106 causes the displayprojection unit 201 to project the second blinking display Pb on thewindshield W (See FIG. 3). The second blinking display Pb is a blinkingdisplay that alerts the driver to the preceding vehicle Nb.

If it is determined that there is the preceding vehicle Nb whosedeceleration is equal to or greater than the deceleration threshold, thewarning control unit 106 recognizes the image of the preceding vehicleNb on the windshield W (the image as viewed by the driver from the eyepoint Ep) based on the recognition result of the another-vehiclerecognition unit 101. For example, the warning control unit 106 uses aknown image processing method (viewpoint conversion processing or thelike) to recognize the image of the preceding vehicle Nb based on theimage captured by the stereo camera 4.

The warning control unit 106 causes the display projection unit 201 toproject the second blinking display Pb in such a manner that thedriver's line of sight is guided toward the preceding vehicle Nb withinthe circular outer display frame C1 shown in FIG. 3. The warning controlunit 106 causes the display projection unit 201 to project the secondblinking display Pb at a position below the image of the precedingvehicle Nb. The inner display frame C2 shown in FIG. 3 is a circularframe that is set to prevent the second blinking display Pb fromoverlapping with the image of the preceding vehicle Nb. The innerdisplay frame C2 is set to surround the lower part of the image of thepreceding vehicle Nb. The warning control unit 106 sets the innerdisplay frame C2 as a larger frame as the distance between the vehicle 2and the preceding vehicle Nb is shorter. The warning control unit 106causes the display projection unit 201 to project the second blinkingdisplay Pb outside the inner display frame C2 in an area in the outerdisplay frame C1 and below the reference horizontal line D. The innerdisplay frame C2 is not projected on the windshield W, and the innerdisplay frame C2 need not necessarily be set.

The warning control unit 106 may cause the display projection unit 201to project the second blinking display Pb using the following method. Inthe description below, consider the state in which the vehicle 2 isviewed from the side as shown in FIG. 2A. The height Eh of the driver'seye point Ep and the distance Lp from the driver's eye point Ep to thefront end of the vehicle 2, both of which are shown in FIG. 2A, arecharacteristic values determined, for example, by vehicle type. Thedifference in the appearance (difference in scale) between the image ofthe preceding vehicle Nb as viewed from the driver's eye point Ep andthe image of the preceding vehicle Nb as viewed from the windshield W isdetermined by the characteristic values described above. The scale ofthe image of the preceding vehicle Nb is changed according to thedistance from the vehicle 2 to the preceding vehicle Nb and according tothe characteristic values described above.

FIG. 2B is a diagram showing the depression angle θ when a lower end Ntof the preceding vehicle Nb is viewed from the driver's eye point Ep.FIG. 2B shows the lower end (lower end of the rear wheels) Nt of thepreceding vehicle Nb, a distance L from the front end of the vehicle 2to the lower end of the preceding vehicle Nb in the longitudinaldirection of the vehicle 2, a straight line Hn joining the driver's eyepoint Ep and the lower end Nt of the preceding vehicle Nb, and the angleθ formed by the straight line Hn and the straight line Hp. The straightline Hp is a straight line passing through the driver's eye point Ep andextending in the longitudinal direction of the vehicle 2. The lower endNt of the preceding vehicle Nb is the lower end of the preceding vehicleNb in the three-dimensional space. The lower end Nt of the precedingvehicle Nb can be identified by known image processing, for example,based on the image captured by the stereo camera 4. Furthermore, thedistance L from the front end of the vehicle 2 to the lower end Nt ofthe preceding vehicle Nb in the longitudinal direction of the vehicle 2can be detected, for example, based on the image captured by the stereocamera 4 (captured image including depth information) or the obstacleinformation sensed by the radar sensor 5. Alternatively, theinter-vehicle distance between the vehicle 2 and the preceding vehicleNb may be used simply as the distance L. The angle θ between thestraight line Hn and the straight line Hp corresponds to the depressionangle formed when the driver views the lower end Nt of the precedingvehicle Nb from the driver's eye point Ep. The depression angle θ can becalculated, for example, by the following equation (1).θ=Tan−1{Eh/(L+Lp)}  (1)

On the other hand, as shown in FIG. 2A, the angle between the straightline Hu, which joins the driver's eye point Ep and the upper end of thesecond blinking display Pb, and the straight line Hp is defined as anangle (depression angle) θe. In this case, the warning control unit 106causes the display projection unit 201 to project the second blinkingdisplay Pb on the windshield W so that the depression angle θ<depressionangle θe. That is, the warning control unit 106 causes the displayprojection unit 20:1 to project the second blinking display Pb so thatthe depression angle θe, formed when the driver views the upper end ofthe second blinking display Pb, is larger than the depression angle θformed when the driver views the lower end Nt of the preceding vehicleNb from the driver's eye point Ep (a larger downward angle withreference to the straight line Hp). The warning control unit 106determines the position of the upper end of the second blinking displayPb (the position in the vertical direction) so that the depression angleθ< the depression angle θe, for example, based on the depression angle θcalculated from the above equation (1). Also, the warning control unit106 determines the position of the second blinking display Pb in thehorizontal direction based on the image of the preceding vehicle Nb. Inthis case, the warning control unit 106 can cause the display projectionunit 201 to project the second blinking display Pb at a position belowthe image of the preceding vehicle Nb based on the upper-end positionand the position in the horizontal direction, which have been determinedas described above, of the second blinking display Pb.

Furthermore, if it is determined by the first determination unit 107that the driver is not facing forward while a light stimulus is beingpresented, the warning control unit 106 causes the warning unit 20 toperform a first warning. In this embodiment, the first warning is asound output by the sound output unit 202. The warning control unit 106outputs a signal to the warning unit 20 to control the operation of thesound output unit 202 of the warning unit 20, thereby outputting thesound in the interior of the vehicle. In this way, the warning controlunit 106 can stimulate the hearing sense of a driver who is not racingforward.

The vehicle system 1 is configured by the warning control unit 106,first determination unit 107, and warning unit 20 described above.

[Processing Content of ECU 10]

The processing performed by the ECU 10 will be described.

[Light Stimulus Presentation Start Processing]

First, the processing of starting the presentation of a light stimuluswill be described. FIG. 6 is a flowchart of the light stimuluspresentation start processing. The flowchart shown in FIG. 6 isperformed by the ECU 10, for example, when the display control button isturned on by the driver.

The ECU 10 performs the traffic environment acquisition processing (S10)to acquire the traffic environment ahead of the vehicle 2. Next, the ECU10 performs the change determination processing (S12) to determinewhether a change in the traffic environment ahead of the vehicle 2 isdetected. If the preceding vehicle of the vehicle 2 decelerates at arate equal to or higher than the deceleration threshold or if there is avehicle that is going squeeze in front of the vehicle 2, the ECU 10detects a change in the traffic environment. If a change in the trafficenvironment is detected by the change determination processing (S12),the warning control unit 106 of the ECU 10 performs the light stimuluspresentation start processing (S14) to cause the display projection unit201 to project a light stimulus.

If a light stimulus is projected by the display projection unit 201 inthe light stimulus start processing (S14) or if a change in the trafficenvironment is not detected in the change determination processing(S12), the ECU 10 terminates the flowchart shown in FIG. 6. If a lightstimulus is not being presented and if the display control button is notturned off by the driver, the ECU 10 performs the flowchart shown inFIG. 6 from the beginning. If a light stimulus is being presented or ifthe display control button is turned off by the driver, the ECU 10 doesnot repeatedly perform the flowchart shown in FIG. 6. The ECU 10performs the flowchart shown in FIG. 6 to present a light stimulus tothe driver according to the change in the environmental.

[Light Stimulus Presentation Termination Processing]

Next, the light stimulus presentation termination processing will bedescribed. FIG. 7 is a flowchart of the light stimulus presentationtermination processing. The flowchart shown in FIG. 7 is performed bythe ECU 10, for example, after a light stimulus is presented.

The ECU 10 performs the traffic environment acquisition processing (S20)to acquire the traffic environment ahead of the vehicle 2. Next, the ECU10 performs the traffic environment change end determination processing(S22) to determine whether the change in the traffic environment aheadof the vehicle 2 has ended. If a predetermined time has elapsed afterthe preceding vehicle of the vehicle 2 started decelerating at a rateequal to or higher than the deceleration threshold or if the squeezingin front of the vehicle 2 is completed, the ECU 10 determines that thechange in the traffic environment has ended.

If the traffic environment change has ended, the warning control unit106 of the ECU 10 performs the light stimulus presentation terminationprocessing (S24) to terminate the presentation of the light stimulus.After that, the ECU 10 terminates foe flowchart shown in FIG. 7. If thepresentation of the light stimulus is terminated, the ECU 10 does notrepeatedly perform the flowchart shown in FIG. 7.

On the other hand, if it is determined in the traffic environment changeend determination processing that the change in the traffic environmenthas not ended (S22), the warning control unit 106 performs the faceorientation determination processing (S26) to determine Whether thedriver is facing forward. If it is recognized that the driver's face isoriented to the front based on the result captured by the driver monitorcamera 6, the warning control unit 106 determines that the driver isfacing forward. If it is recognized that the driver's face is notoriented to the front based on the result captured by the driver monitorcamera 6, the warning control unit 106 determines that the driver is notfacing forward.

If it is determined in the face orientation determination processing(S26) that the driver is facing forward, the warning control unit 106performs the waiting processing (S28) to wait for a predetermined timeand then proceeds to the light stimulus presentation terminationprocessing (S24). By waiting for a predetermined time, the warningcontrol unit 106 prevents the presentation of a light stimulus frombeing finished so quickly that the light stimulus cannot be recognizedby the driver. The processing performed after the light stimuluspresentation termination processing (S24) is the same as that describedabove.

If it is determined in the face orientation determination processing(S26) that the driver is not facing forward, the warning control unit106 terminates the flowchart shown in FIG. 7. If a light stimulus isbeing presented, the ECU 10 performs the flowchart, shown in FIG. 7,from the beginning.

As described above, the presentation of a light stimulus is terminatedwhen the change in the traffic environment ahead of the vehicle 2 hasended or when the driver is facing forward.

[First Warning Start Processing]

Next, the first warning start processing will be described. FIG. 8 is aflowchart of the first warning start processing. The flowchart shown inFIG. 8 is performed by the ECU 10, for example, after presenting a lightstimulus.

The warning control unit 166 of the ECU 10 performs the race orientationdetermination processing (S30) to determine whether the driver is facingforward. This processing is the same as the face orientationdetermination processing (S26).

If it is determined in the face orientation determination processing(S30) that the driver is facing forward, the warning control unit 106terminates the flowchart shown in FIG. 8. If a light stimulus is beingpresented and if the first warning is not being performed, the ECU 50performs the flowchart, shown in FIG. 8, from the beginning.

If it is determined in the face orientation determination processing(S30) that the driver is not facing forward, the warning control unit106 performs the warning start processing (S32) to cause the warningunit 20 to start the first warning. The sound output unit 202 of thewarning unit 20 outputs a sound as the first warning. When the warningstart processing (S32) is terminated, the ECU 10 terminates theflowchart shown in FIG. 8, When the first warning is started, the ECU 10does not repeatedly perform the flowchart shown in FIG. 8.

[First Warning Termination Processing]

Next, the first warning termination processing will be described. FIG. 9is a flowchart of the first warning termination processing. Theflowchart shown in FIG. 9 is performed by the ECU 10, for example, afterthe first warning is started.

The warning control unit 106 of the ECU 10 performs the face orientationdetermination processing (S40) to determine whether the driver is facingforward. This processing is the same as the face orientationdetermination processing (S26).

If it is determined in the face orientation determination processing(S40) that the driver is facing forward, the warning control unit 106performs the warning termination processing (S42) to cause the warningunit 20 to terminate the first warning. Upon completion of the warningtermination processing (S42), the ECU 10 terminates the flowchart shownin FIG. 9. When the first warning is terminated, the ECU 10 does notrepeatedly perform the flowchart shown in FIG. 9.

If it is determined in the face orientation determination processing(S40) that the driver is not facing forward, the ECU 10 terminates theflowchart shown in FIG. 9. If the first warning is being performed, theECU 10 performs the flowchart, shown in FIG. 9, from the beginning.

[Effect of the Vehicle System 1]

In the vehicle system 1, when a change in the traffic environment aheadof the vehicle 2 is detected during antonomous driving, the head updisplay of the vehicle 2 is used to present a light stimulus to thedriver of the vehicle 2. A light stimulus, presented in this way,prompts the driver who is aware of driving to pay attention ahead. If itis determined by the first determination unit 107 that the driver is notfacing forward while a light stimulus is being presented, the warningunit 20 performs the first warning that is a sound output by the soundoutput unit 202. A sound, output in this way, allows the vehicle system1 to give a stimulus also to a non-visual sense of the driver who is notfacing forward while a light stimulus is being presented, thusencouraging the driver to face forward. In this way, the vehicle system1 can encourage the driver to pay attention to driving according to thestate of the driver.

Second Embodiment

A vehicle system 1A according to a second embodiment is different fromthe vehicle system 1 according to the first embodiment in that avibration generation unit 203 is provided instead of the sound outputunit 202. The other components are the same as those of the firstembodiment. In the description of the vehicle system 1A, the descriptionsimilar to that of the vehicle system 1 is omitted.

FIG. 10 is a functional block diagram of the vehicle 2 having thevehicle system 1A according to the second embodiment. As shown in FIG.10, the difference in the configuration between the vehicle system 1Aand the vehicle system 1 is that the vehicle system 1A does not includethe sound output unit 202 but includes the vibration generation unit203. The other configuration is the same as that of the vehicle system1.

The vibration generation unit 203 is a device that generates avibration. For example, the vibration generation unit 203 has a weighthaving a biased center of gravity and a motor for rotating the weight.The vibration generation unit 203 may use a known structure. Thevibration generation unit 203 is arranged in the interior of thevehicle. More specifically, the vibration generation unit 203 isprovided in the steering wheel, the seat, the armrest, and the like.

When performing the first warning, the vehicle system 1A uses avibration generated by the vibration generation unit 203. The otherprocessing of the vehicle system 1A is identical to that of the vehiclesystem 1.

[Effect of the Vehicle System 1A]

In the vehicle system 1A, when a change in the traffic environment aheadof the vehicle 2 is detected during autonomous driving, the head-updisplay of the vehicle 2 is used to present a light stimulus to thedriver of the vehicle 2. A light stimulus, presented in this way,prompts the driver who is aware of driving to pay attention ahead. Inaddition, in the vehicle system 1A, if it is determined by the firstdetermination unit 107 that the driver is not facing forward while alight stimulus is being presented, the warning unit 20 performs a firstwarning that is a vibration generated by the vibration generation unit203. A vibration, generated in this way allows the vehicle system 1A togive a stimulus also to a non-visual sense of the driver Who is notfacing forward while a light stimulus is being presented, thusencouraging the driver to face forward. In this way the vehicle system1A can encourage the driver to pay attention to driving according to thestate of the driver.

Third Embodiment

A vehicle system 1B according to a third embodiment is different fromthe vehicle system 1 according to the first embodiment in that a seconddetermination unit 108 and a vibration generation unit. 203 are providedand in the processing content of a warning control unit 106B. The othercomponents and processing content are the same as those of the firstembodiment. In the description of the vehicle system 1B, the descriptionsimilar to that of the vehicle systems 1 and 1A is omitted.

FIG. 11 is a functional block diagram of the vehicle 2 having thevehicle system 1B according to the third embodiment. As shown in FIG.11, the difference in the configuration between the vehicle system 1Band the vehicle system 1 is that the vehicle system 1B includes thewarning control unit 106B, the function of which is partially differentfrom that of the warning control unit 106, the second determination unit108, and the vibration generation unit 203. The other configuration isthe same as that of the vehicle system 1.

The second determination unit 108 determines whether the driver holdsthe steering wheel. For example, the second determination unit 108determines whether the driver holds the steering wheel based on theresult sensed by the pressure sensor provided in the steering wheel orbased on the result captured by the driver monitor camera 6. The seconddetermination unit 108 outputs the determination result to the warningcontrol unit 106B.

When performing the first warning, the warning control unit 106B of thevehicle system 1B causes the warning unit 20 to perform the firstwarning that is either a vibration generated by the vibration generationunit 203 or a sound output by the sound output unit 202.

The warning control unit 106B causes the warning unit 20 to perform asecond warning while a light stimulus is being presented if it isdetermined by the first determination unit 107 that the driver is notfacing forward and if it is determined by the second determination unit108 that the driver does not hold the steering wheel before apredetermined time elapses after the first warning is started. Thesecond warning is a warning that includes both a vibration, generated bythe vibration generation unit 203, and a sound output by the soundoutput unit 202. The warning control unit 106B provides two warnings, afirst warning and a second warning, and switches the warning accordingto the level of the drivers driving awareness. The warning control unit106B determines a driver who does not turn the face toward the front inresponse to a light stimulus but is holding (or once held) the steeringwheel as not being a driver whose wakefulness level is extremely low oras a driver whose driving awareness level is extremely low. On the otherhand, the warning control unit 106B determines a driver who does notturn the face toward the front in response to a light stimulus and isnot holding the steering wheel as being a driver whose wakefulness levelis extremely low or as a driver whose driving awareness level isextremely low. For such a driver, the warning control unit 106B outputsthe second warning stronger than the first warning. The second warningis a warning that has a warning level higher than that of the firstwarning or a warning that excites more senses than the first warning.

[First Warning Termination Processing and Second Warning StartProcessing]

Next, the second warning start processing will be described. FIG. 12 isa flowchart showing the second warning start processing. The flowchartshown in FIG. 12 is performed by the ECU 10, for example, after thestart of the first warning. The flowchart shown in FIG. 12 also includesthe first warning termination processing.

The warning control unit 106B of the ECU 10 performs the faceorientation determination processing (S80) to determine whether thedriver is facing forward. This processing is the same as the faceorientation determination processing (S26).

If it is determined in the face orientation determination processing(S80) that the driver is facing forward, the warning control unit 106Bperforms the warning termination processing (S82) to cause the warningunit 20 to terminate the first warning. This processing is the same asthe warning termination processing (S42). When the warning terminationprocessing (S82) is completed, the ECU 10 terminates the flowchart shownin FIG. 12. When the first warning is terminated, the ECU 10 does notrepeatedly perform the flowchart shown in FIG. 12.

If it is determined in the face orientation determination processing(S80) that the driver is not facing forward, the second determinationunit 108 of the ECU 10 performs the first steering wheel holding,determination processing (S84) to determine whether the driver isholding the steering wheel.

If it is determined in the first steering wheel holding determinationprocessing (S84) that the driver is holding the steering wheel, the ECU10 performs the flowchart, shown in FIG. 12, from the beginning.

On the other hand, if it is determined in the first steering wheelholding determination processing (S84) that the driver is not holdingthe steering wheel, the warning control unit 106B performs the warningtermination processing (S86) to cause the warning unit 20 to terminatethe first warning. The warning termination processing (S86) warningcontrol unit 106B performs the warning start processing (S88) to causethe warning unit 20 to perform the second warning that includes avibration, generated by the vibration generation unit 203, and a soundoutput by the sound output unit 202. When the first warning isterminated and the second warning is started, the ECU 10 does notrepeatedly perform the flowchart shown in FIG. 12.

[Second Warning Termination Processing]

Next, the second warning termination processing will be described. FIG.13 is a flowchart showing the second warning termination processing. Theflowchart shown in FIG. 13 is performed by the ECU 10, for example,after the start of the second warning.

The warning control unit 106B of the ECU 10 performs the faceorientation determination processing (S90) to determine whether thedriver is facing forward. This processing is the same as the faceorientation determination processing (S26).

If it is determined in the race orientation determination processing(S90) that the driver is facing forward, the warning control unit 106Bperforms the warning termination processing (S92) to cause the warningunit 20 to terminate the second warning. When the warning terminationprocessing (S92) is completed, the ECU 10 terminates the flowchart shownin FIG. 13. When the second warning is terminated, the ECU 10 does notrepeatedly perform the flowchart shown in FIG. 13.

If it is determined in the face orientation determination processing(S90) that the driver is not facing forward, the second determinationunit 108 of the ECU 10 performs the second steering wheel holdingdetermination processing (S96) to determine whether the driver isholding the steering wheel.

If it is determined in the second steering wheel holding determinationprocessing (S96) that the driver is holding the steering wheel, thewarning control unit 106B performs the warning termination processing(S92) to cause the warning unit 20 to terminate the second warning. Thesubsequent processing is the same as that described above.

On the other hand, if it is determined in the second steering wheelholding determination processing (S96) that the driver is not holdingthe steering wheel, the ECU 10 terminates the flowchart shown in FIG.13. The ECU 10 performs the flowchart, shown in FIG. 13, from thebeginning.

[Effect of the Vehicle System 1B]

The vehicle system 1B can give the second warning, which is strongerthan the first warning, to a driver whose wakefulness level is furtherreduced or to a driver who relies on autonomous driving so much that theinterest in driving is further reduced.

Fourth Embodiment

A vehicle system 1C according to a fourth embodiment is different fromthe vehicle system 1B according to the third embodiment in that avehicle control unit 109 is provided and in the processing content of awarning control unit 106C. The other components and processing contentare the same as those of the third embodiment. In the description of thevehicle system 1C, the description similar to that of the vehiclesystems 1, 1A, and 1B is omitted.

FIG. 14 is a functional block diagram of the vehicle 2 having thevehicle system 1C according to the fourth embodiment. As shown in FIG.14, the difference in the configuration between the vehicle system 1Cand the vehicle system 1B is that the vehicle system IC includes thewarning control unit 106C, the function of which is partially differentfrom that of the warning control unit 106, and the vehicle control unit109. The other configuration is the same as that of the vehicle system1B.

The warning control unit 106C increments the number of re-holding timesif it is determined by the first determination unit 107 that the driverfaces forward before a predetermined time elapses after the secondwarning is started or if it is determined by the second determinationunit 108 that the driver holds the steering wheel before a predeterminedtime elapses after the second warning is started. The number ofre-holding times refers to the number of times the driver re-holds thesteering wheel in response to the second warning. There is a possibilitythat a driver who receives the warning many times is overconfident inautonomous driving. With this possibility in mind, the warning controlunit 106C increments the number of re-holding times, stored in thestorage unit, for use in identifying a driver who receives the warningwhile a light stimulus is being presented and, in response to thewarning, re-holds the steering wheel in a hurry before the vehicle 2 isforced to stop.

The vehicle control unit 109 controls the operation of the vehicle 2. Ifthe number of re-holding times while a light stimulus is being presentedis equal to or greater than the threshold, the vehicle control unit 109outputs the signal for terminating the autonomous driving of the vehicle2 to the autonomous driving ECU 3 of the vehicle 2. In this way, adriver who is overconfident in autonomous driving is prohibited fromdriving in the autonomous mode.

In addition, if it is determined by the second determination unit 108that the driver does not hold the steering wheel before a predeterminedtime elapses after the second warning is started, the vehicle controlunit 109 decelerates or stops the vehicle 2. A driver who is not awakeis response to the second warning is considered a driver with anextremely low wakefulness level. In this case, the vehicle control unit109 operates various actuators to perform the emergency processing fordecelerating or stopping the vehicle 2.

[Light Stimulus Presentation Termination Processing and Count ResettingProcessing]

The number of re-holding times, which is the number of re-holding timeswhile a light stimulus is being presented, is reset when thepresentation of the light stimulus is finished. To help understand thisprocess, the light stimulus presentation termination processing and thecount resetting processing will be described below. FIG. 15 is aflowchart showing the light stimulus presentation termination processingand the count resetting processing. The flowchart shown in FIG. 15 isperformed by the ECU 10, for example, after the presentation of a lightstimulus.

The traffic environment acquisition processing (S120), the terminationdetermination processing (S122), the light stimulus presentationtermination processing (S124), the face orientation determinationprocessing (S126), and the waiting processing (S128) correspondrespectively to the traffic environment acquisition processing (S20),the termination determination processing (S22), the light stimuluspresentation termination processing (S24), the face orientationdetermination processing (S26), and the waiting processing (S28) shownin FIG. 7.

If the traffic environment change has ended, the warning control unit106 C of the ECU 10 performs the light stimulus presentation terminationprocessing (S124) to terminate the presentation of a light stimulus.After that, the warning control unit 106C of the ECU 10 performs theresetting processing (S125) to reset the number of re-holding timesstored in the storage unit. After that, the ECU 10 terminates theflowchart shown in FIG. 15. The other processing is the same as that inthe flowchart shown in FIG. 7.

As described above, the presentation of a light stimulus is terminatedwhen the change in the traffic environment ahead of the vehicle 2 hasended or when the driver is facing forward. In addition, the number ofre-holding times is reset when the presentation of a light stimulus isterminated.

[Second Warning Termination Processing, Counting Processing, andEmergency Processing]

Next, the second warning termination processing, counting processing,and emergency processing will be described. FIG. 16 is a flowchartshowing the second warning termination processing, counting processing,and emergency processing. The flowchart shown in FIG. 16 is performed bythe ECU 10, for example, after the start of the second warning.

The face orientation determination processing (S190), the warningtermination processing (S192), and the second steering wheel holdingdetermination processing (S194) correspond respectively to the faceorientation determination processing (S90), the warning terminationprocessing (S92) and the second steering wheel holding determinationprocessing (S96) shown in FIG. 13.

If it is determined in the second steering wheel holding determinationprocessing (S194) that the driver is holding the steering wheel, thewarning control unit 106C performs the counting processing (S196) toincrement the count value stored in the storage unit. After that, thewarning control unit 106C performs the warning termination processing(S192) to cause the warning unit 20 to terminate the second warning. Thesubsequent processing is the same as that described above.

On the other hand, if it is determined in the second steering wheelholding determination processing (S194) that the driver is not holdingthe steering wheel, the ECU 10 performs the deceleration notificationprocessing (S198) to notify the driver that the vehicle 2 will beemergency decelerated or emergency stopped. After that, the vehiclecontrol unit 109 performs the deceleration/stop processing (S199) todecelerate or stop the vehicle 2. Then, the ECU 10 terminates theflowchart shown in FIG. 16. The ECU 10 does not repeatedly perform theflowchart shown in FIG. 16.

[Autonomous Driving Termination Processing]

FIG. 17 is a flowchart showing the autonomous driving terminationprocessing. The flowchart shown in FIG. 17 is performed by the ECU 10,for example, while a light stimulus is being presented.

The vehicle control unit 109 of the vehicle system 1C performs thenumber-of-times determination processing (S200) to determine whether thenumber of re-holding times is equal to or greater than the threshold. Ifit is determined in the number-of-times determination processing (S200)that the number of re-holding times is equal to or greater than thethreshold, the vehicle control unit 109 performs the notificationprocessing (S202) to notify in advance that autonomous driving will beterminated. After that, the vehicle control unit 109 performs theautonomous driving termination processing (S204) to output the signal tothe autonomous driving ECU 3 for terminating the autonomous driving ofthe vehicle 2. The autonomous driving ECU 3 terminates the autonomousdriving of the vehicle 2.

If it is determined in the number-of-times determination processing(S200) that the number of re-holding times is not equal to or greaterthan the threshold or if the autonomous driving, termination processing(S204) is terminated, the ECU 10 terminates the flowchart shown in FIG.17. If a light stimulus is being presented and if the autonomous drivingtermination processing (S204) is not performed, the ECU 10 repeatedlyperforms the flowchart shown in FIG. 17.

[Effect of the Vehicle System 1C]

In the vehicle system 1C, if the number of re-holding times is equal toor greater than the threshold while a light stimulus is being presented,the vehicle control unit 109 outputs the signal to the autonomousdriving ECU 3 of the vehicle 2 to terminate the autonomous driving ofthe vehicle 2. In this way, for a driver who has a tendency not to payattention to driving, the vehicle system 1C can terminate the autonomousdriving of the vehicle 2, thus forcing the driver to drive the vehiclehimself or herself.

In addition, if it is determined by the second determination unit 108that the driver does not hold the steering wheel before a predeterminedtime elapses after the second warning is started, the vehicle system ICcan decelerate or stop the vehicle 2, thus ensuring traveling safety.

The above-described embodiments can be implemented by adding variouschanges and improvements based on the knowledge of those skilled in theart. For example, the embodiments may be combined.

Example of Configuration Modifications

A monocular camera may be used instead of the stereo camera 4. Thestereo camera 4 may be provided to capture the left and right sides ofthe vehicle 2. The radar sensor 5 may be provided to detect an obstacleon the left and right sides of the vehicle 2. A plurality of drivermonitor cameras 6 may be provided for capturing the driver Dr from aplurality of directions.

The ECU 10 may be configured by a plurality of electronic control units,A part of the functions of the ECU 10 may be executed by a computer offacilities, such as an information management center capable ofcommunicating with the vehicle 2 or may be executed by a portableinformation terminal capable of communicating with the vehicle 2.

The display projection unit 201 can use a configuration known as theconfiguration of a head-up display. The display projection unit 201 maybe any device that projects onto the display area fixed in the frontpart of the vehicle. For example, the display projection unit 201 may bean embedded head-up display embedded in dashboard of the vehicle 2.

The vehicle 2 does not have to include the autonomous driving ECU 3. Inthe above embodiments, the vehicle 2 has the function to driveautonomously. However, the present disclosure may be applied not only toa vehicle that can drive autonomously. For example, the above-describedembodiments may be applied also to a vehicle that does not have theautonomous driving function.

[Modification of Processing]

In the fourth embodiment, the counting processing, the resettingprocessing, and the autonomous driving termination processing may beomitted. That is, the vehicle system 1C may perform only the emergencyprocessing.

What is claimed is:
 1. A vehicle system comprising: a warning deviceincluding at least one of a vibration generation device configured togenerate a vibration and a sound output device configured to output asound, and a display projection device configured to project a displayon a display area of a head-up display provided in a vehicle; and atleast one electronic control unit configured to present a light stimulusto a driver of the vehicle by using the head-up display when a change ina traffic environment is detected ahead of the vehicle, while thevehicle is performing autonomous driving, determine whether the driveris facing forward, determine whether the driver holds a steering wheel,and cause the warning device to perform a first warning, when it isdetermined that the driver is not facing forward while the lightstimulus is being presented, the first warning being either thevibration generated by the vibration generation device or the soundoutput by the sound output device, and perform a second warning, when itis determined that the driver does not hold the steering wheel before afirst predetermined time elapses after the first warning is started, thesecond warning including the vibration generated by the vibrationgeneration device and the sound output by the sound output device. 2.The vehicle system according to claim 1, wherein the at least oneelectronic control unit is configured to increment a number ofre-holding times, when it is determined that the driver faces forwardbefore a second predetermined time elapses after the second warning isstarted or when it is determined that the driver holds the steeringwheel before the second predetermined time elapses after the secondwarning is started, and output a signal for terminating the autonomousdriving of the vehicle to an autonomous driving electronic control unit,when number of re-holding times while the light stimulus is beingpresented, is equal to or greater than a threshold, the autonomousdriving electronic control unit being provided in the vehicle and beingconfigured to perform the autonomous driving of the vehicle.
 3. Thevehicle system according to claim 2, wherein the at least one electroniccontrol unit is configured to decelerate the vehicle or stop the vehiclewhen it is determined that the driver does not hold the steering wheelbefore the second predetermined time elapses after the second warning isstarted.
 4. The vehicle system according to claim 1, wherein the atleast one electronic control unit is configured to decelerate thevehicle or to stop the vehicle when it is determined that the driverdoes not hold the steering wheel before a second predetermined timeelapses after the second warning is started.
 5. The vehicle systemaccording to claim 1, further comprising a driver monitor camera,wherein the at least one electronic control unit is configured to detectwhether the driver is facing forward by using the driver monitor camera.6. The vehicle system according to claim 1, wherein the second warninghas a stimulus intensity higher than the stimulus intensity of the firstwarning.
 7. The vehicle system according to claim 1, wherein thevibration generation device includes a weight and motor configured torotate the weight to generate the vibration, and wherein the soundoutput device includes a speaker configured to output the sound.
 8. Avehicle comprising: a sensor configured to detect a change in a trafficenvironment ahead of the vehicle; a head-up display including a displayprojection device, the display projection device being configured topresent a light stimulus to a driver of the vehicle by projecting adisplay on a display area of the head-up display based on a detectionresult of the sensor; a warning device including at least one of avibration generation device and a sound output device, the warningdevice configured to output at least one of a vibration generated by thevibration generation device and a sound output by the sound outputdevice; and at least one electronic control unit configured to cause thedisplay projection device to present the light stimulus, determinewhether the driver is facing forward, cause the warning device toperform a first warning when it is determined that the driver is notfacing forward while the light stimulus is being presented, determinewhether the driver holds a steering wheel, and cause the warning deviceto perform a second warning, when it is determined that the driver doesnot hold the steering wheel before a first predetermined time elapsesafter the first warning is started, cause the warning device to performa second warning, when it is determined that the driver does not holdthe steering wheel before a first predetermined time elapses after thefirst warning is started, wherein the second warning: has a stimulusintensity higher than the stimulus intensity of the first warning, or isperformed to stimulate a number of senses of the driver greater than anumber of senses that the first warning is performed to stimulate. 9.The vehicle according to claim 8, further comprising an autonomousdriving electronic control unit configured to perform autonomous drivingof the vehicle, wherein the at least one electronic control unit isconfigured to cause the autonomous driving electronic control unit toperform the autonomous driving of the vehicle, increment the number ofre-holding times when it is determined that the driver faces forwardbefore a second predetermined time elapses after the second warning isstarted or when it is determined that the driver holds the steeringwheel before the second predetermined time elapses after the secondwarning is started, and output a signal for terminating the autonomousdriving of the vehicle to the autonomous driving electronic control unitwhen the number of re-holding times is equal to or greater than athreshold.
 10. The vehicle according to claim 8, wherein the at leastone electronic control unit is configured to decelerate the vehicle orto stop the vehicle when it is determined that the driver does not holdthe steering wheel before a second predetermined time elapses after thesecond warning is started.
 11. The vehicle according to claim 8, whereinthe senses of the driver include a visual sense and a non-visual sense.12. The vehicle according to claim 8, wherein the vibration generationdevice includes a weight and motor configured to rotate the weight togenerate the vibration, and wherein the sound output device includes aspeaker configured to output the sound.